Doctoral defence: Martin Oja „Semiotics of Multimodal Conflict: Modes, Modalities and Audio-Visual Incongruence in Feature Film“

On 16 December at 14:15 Martin Oja will defend his doctoral thesis “Semiotics of Multimodal Conflict: Modes, Modalities and Audio-Visual Incongruence in Feature Film” for obtaining the degree of Doctor of Philosophy (in Semiotics and Culture Studies).

Supervisor: 
Professor Peeter Torop, University of Tartu

Opponents: 
Professor Kay O’Halloran, University of Liverpool (Great Britain)
Research Professor Pia Tikka, Tallinn University (Estonia)

Summary
This dissertation aspires to contribute to the development of the semiotic theory of multimodality. Primarily, it addresses the central concepts of the theory – (semiotic) mode and (sensory) modality. Various authors use these terms interchangeably and without sufficient specification of their meanings. To overcome such a problem, the dissertation proposes a two-level framework that combines the constancy of modalities and the flexibility of modes, seeing modalities and modes in a mutually complementary relationship. The organization of sensory modalities can be seen as a foundational layer under the system of modes. 

The second objective of the dissertation is to construct a typology of modality relations. This typology distinguishes supporting, modifying, conflicting, substituting, and cross-activating relations. Subsequently, the work focuses on conflicting modalities as an extremely interesting and revealing category that corresponds to several semiotic strategies and purposes, e.g., the artistic devices that defamiliarize or self-reflexively indicate the constructedness of the text. Also, conflicting modalities are occasionally employed in deceptive meaning-making. An instance of such strategy is discussed, analyzing film trailers that are used to communicate one genre by auditive and another genre by visual modality. 

The dissertation proposes that synthetic meaning-making can be examined in light of Juri Lotmanʼs idea about two incompatible languages that comprise a mechanism for creating new information. Discussing multimodal conflicts in various feature films, the work focuses on some specific devices, such as contrapuntal editing and the use of internal diegetic sound. The possibilities of detecting and measuring the presence of multimodal conflict are also evaluated. Redundancy and synchrony as possible parameters that are violated in multimodal conflict are considered but dismissed. Instead, the work acknowledges that space as the fundamental device for artistic modeling in Juri Lotmanʼs semiotics allows the modeling of multimodal conflicts. The alternation between conflicts and habituation is seen as a fundamental component of meaning-making both in nature and culture.

The defence will be held also via Zoom (meeting ID: 924 4360 4410, passcode: 914783).